The Christmas Bird Count (CBC), conducted by the National Audubon Society (NAS) since 1900, constitutes the longest-running and geographically most widespread survey of bird life in the Western Hemisphere. Starting with 25 count locations in its first year, the program has grown continuously ever since, with 20–30 locations currently being added annually. Each CBC consists of a tally of all birds detected within a circle 24.1 km (15 miles) in diameter, on a single day within a few weeks around Christmas (current allowable dates are 14 December through 5 January). More than 50,000 observers now take part each year, in close to 2,000 count circles spread across the U.S. and its territories, southern Canada, and, increasingly, Latin America.

Christmas Bird Count data have been used in hundreds of publications on a wide range of topics. Many papers describe general biogeographic patterns of bird distribution and abundance in winter (Bock et al. 1978, Bock and Ricklefs 1983, Bock 1984, Root 1988a). Others focus on changes in bird life over time, including changes in distribution (Bock and Lepthien 1976a, Root and Weckstein 1994, Duncan 1996, Pranty 2002) and abundance (Graber and Golden 1960, Bock and Lepthien 1976b, Lepthien and Bock 1976, Yunick 1988, Brennan and Morrison 1991, Sauer et al. 1996). Studies of broad-scale population irruptions are often based on CBC data (Bock and Lepthien 1976c, Bock 1982, Yunick 1984, Davis and Morrison 1987, Smith and Scarlett 1987). Increasingly, CBC data are used to test specific hypotheses about causes and patterns of population change (Dunning and Brown 1982, Hagan 1993, Allen et al. 1995, Viverette et al. 1996, Hochachka and Dhondt 2000, Bonter and Hochachka 2003, Caffrey and Peterson 2003).

Although the number of publications using CBC data is impressive, a scan of the total list on the CBC website reveals that a large proportion appeared in regional or Audubon publications and in government reports. Of those papers published in leading scientific journals, many focus on general spatial patterns and species richness rather than on quantitative analyses of abundance or trend in population size. There are good reasons for that. From early on, the CBC has been recognized as a valuable source of data, but analysis and interpretation are limited by the fact that the program was neither intended nor designed for population monitoring. For example, Stewart (1954) discussed the consequences of there being no rules for how CBCs should be conducted. He listed numerous factors that affect numbers of birds counted within and between circles, including variability in number of participants, hours in the field, extent and modes of travel, coverage of different habitats, skill levels, and use of attractive devices (e.g. bird feeders, noises such as “pishing”). The same problems have been reiterated and discussed in subsequent review articles (Raynor 1975, Arbib 1981, Bock and Root 1981, Sauer and Link 2002). An additional problem for scientific use of CBC data is that spatial distribution of count circles is far from random (Drennan 1981, Sauer and Link 2002). Possibly the most succinct summary of the scientific value of the CBC comes from Bock and Root (1981:17): “The Christmas Bird Count (CBC) is an enormous but weakly standardized avian count…. CBC data are an inappropriate substitute for more controlled census work associated with local projects. Scientists would ignore CBC data altogether, were it not for their potential application to large-scale studies.”

That “potential application,” however, is substantial indeed. No other data set provides such broad temporal and geographic coverage of North America's bird life. The Christmas Bird Count has undoubted value for documenting broad patterns of change in winter distribution (both annual and long-term) and for studies of patterns in species richness. Although the number of uncommon species detected on a given count is certain to increase with effort, rare species are often scouted ahead of time, and the emphasis that most participants put on high species totals ensures that the distributional, presence-absence data in the CBC are very good. Moreover, though not designed as a monitoring program, the CBC indicates changes in bird abundance that correlate roughly with those from the North American Breeding Bird Survey (BBS; Butcher et al. 1990, Dunn and Sauer 1997). Similarly, strong correlations have been noted with results from Project FeederWatch, an independent, more standardized winter bird count (Lepage and Francis 2002). For ≈30 of the landbird species that breed in North America, CBC data are the best available source of information on range-wide population trends during the 20th century (Rich et al. 2004). For the most part, however, the vast potential of the CBC data set remains an untapped resource.

In November 2003, NAS convened an international panel of experts (the authors) to undertake a review of the CBC, including all aspects of survey design, data collection, project management, and output products. The primary charge of the panel was to formulate recommendations to NAS on the best means of extracting scientific value from past results and on realistic changes in data collection (if any) that could increase the value of future data. The panel's detailed recommendations have been published in NAS's annual CBC publication (Francis et al. 2004). Here, we provide more detail on the main scientific issues that limit use of CBC data, along with a brief overview of the panel's recommendations.

Challenges of Using Christmas Bird Count Data for Scientific Purposes

For a survey of animal abundance to provide reliable results for use in science and conservation, it must provide unbiased estimates of population attributes for a defined population of interest. Inference to a population can be achieved only when probabilistic sampling is used (e.g. samples are selected with known probabilities from a list of sample sites) and when either (1) every animal present is counted or (2) an estimate is made of the proportion of individuals detected (Thompson et al. 1998, Buckland et al. 2001). Few bird surveys achieve those goals, of course, but the CBC has been the subject of particular criticism (e.g. Sauer 2000), primarily because of two sampling issues: variability of count effort within and among circles, and nonrandom distribution of the circles.

Variability of Count Effort within and among Circles

Individual CBC circles are often covered fairly consistently every year, but there are no rules to ensure this is the case and no documentation to enable assessment of consistency. Circles are often divided into sectors, and a birding party assigned to each one to ensure that all parts of the circle are sampled and to minimize overlap of counts; but within each sector, counts may be conducted very differently.

The proportions of birds present that are actually counted will vary widely within and among circles, depending not only on amount of effort, but on weather, time of day, skill of observers, mode of travel (driving vs. walking), habitat, and distribution of effort among habitats. To the extent that those factors change systematically over time, they can bias any estimates of population change based on CBC data. For example, increase in average birder skill over the past half-century has been shown to influence BBS counts (Sauer et al. 1994), though that may be less an issue for the CBC because observers do not usually need to identify species by song and there is usually a mix of beginner and experienced birders taking part.

Probably the most important problem with the CBC, from the perspective of estimating population trends, is that effort has not remained steady across time, in either the total amount or the type of counting. Not only has the number of circles increased dramatically over 100 years, so too have the numbers of observers within circles (Yunick 1988, Butcher and McCulloch 1990). Moreover, there has been an increase in the proportion of effort expended on nontraditional count methods, such as nocturnal owling, gull counting, counting from boats, and feeder-watching from indoors. Number of birds detected per unit effort differs among count modes, and Dunn (1995) showed that the increase in home-based feeder watching has introduced substantial bias into CBC trend estimates for certain species that frequent feeders.

Several approaches have been used to adjust CBC bird counts for changes in effort, including calculation of birds per circle, birds per party mile, birds per unit of area covered, and birds per observer (Bock and Root 1981). Most investigations of simple correction factors have identified birds per party hour as the most suitable (Raynor 1975), and that is the most commonly used effort-adjustment factor today (Sauer et al. 1996). Nonetheless, it is known that the relationship between number of birds seen and party hours is not linear, and the shape of the relationship is not the same for all species (Butcher and McCulloch 1990; Link and Sauer 1999a, b32; Sauer and Link 2002). For example, species that concentrate in one part of a circle, such as waterfowl in a small area of suitable habitat, are likely to be counted in similar numbers, regardless of total effort within the count circle. Also, sightings of large, conspicuous species, such as raptors, increase with number of observers to a certain point, but further effort yields diminishing returns. As a result, estimates from analyses based on birds per unit effort, which implicitly assume a linear relationship between effort and counts, can sometimes be seriously biased—potentially even more than estimates based on counts unadjusted for effort.

Statistical modeling approaches outlined by Link and Sauer (1999a, b)32 and Sauer and Link (2002), suitable for analyses that involve many CBC circles, have potential to greatly improve species-specific effort adjustments. However, such modeling cannot take into account systematic change in type of effort (e.g. increase in home-based feeder-watching), because birds detected using each type of effort have not been recorded separately in the database. Moreover, effort-adjustment factors based on modeling of counts from many locations may not be suitable for adjusting counts from individual CBC locations, because the relationship between effort and bird numbers can vary among circles.

Nonrandom Distribution of Christmas Bird Count Circles

In the early years of the CBC, count sites were selected near population centers. As travel became easier, more circles were established in bird-rich areas far from cities, but those may be just as unrepresentative of the wider landscape as urban-centered circles, and the density of CBC locations is still correlated with human population density (Drennan 1981).

Purposeful site-selection has two important consequences. First, pooled analyses will be biased toward results from regions where circles are most dense unless steps are taken to weight circles appropriately. Some authors have calculated mean values for latitude-longitude blocks or states before taking overall means (Bock et al. 1978, Hagan 1993), and a few have weighted circles inversely to their density within each portion of the range (Butcher et al. 1990, Sauer et al. 1996). Appropriate stratification or weighting should be a routine feature of broad-scale analyses to adjust for differing density of CBC locations.

The second, more difficult problem posed by nonrandom distribution of CBC locations is that circles may not be representative of the regions that contain them. Unless appropriate adjustment can be made, results may correctly portray bird life within the circles themselves but not reflect population status across the entire landscape. In some parts of the United States, a remarkably high percentage of the land area falls within a CBC circle (e.g. 37% of Maryland; C. S. Robbins pers. comm.), and the fact that each CBC covers a large area means that much “average” landscape may be included within each circle. Nonetheless, research is needed on the representativeness of CBC circles, and on statistical models that can appropriately address the site-selection problems. The increasing availability of digital landcover data derived from remote sensing makes such research and modeling feasible, and these activities are needed to determine the potential value of habitat adjustment in CBC analyses.

Other Problems

An additional limitation to scientific use of CBC data, though not a science issue in itself, is the relative inaccessibility of the data to researchers. As noted by Bock (1980:28), “A major impediment to full and effective utilization of Christmas Count data is the very quantity of information available. The job of extracting and compiling by hand the data on a particular species is so staggering, especially if the species is at all widespread, that few ornithologists can afford the time.” Bock extracted and computerized 10 years of CBC data, which became the basis for more than two dozen publications and an atlas of winter distribution (Root 1988b), providing dramatic evidence of the benefits of maintaining a digital database. Data from a later set of years were entered into computer files and made available for downloading by John Shipman from his webpage, and data for individual routes can be downloaded from NAS's CBC website. Nonetheless, the data set is incomplete (e.g. weather data are missing for some years), there are known errors that have not yet been corrected, and there are many as-yet-unresolved difficulties caused by changes in circle names or locations, and by changes in species' names through taxonomic revision in the past century. Although researchers interested in using CBCs are now able to access much of the data, the task of editing the database to make it fully usable is still substantial, and adjusting counts for effort remains the responsibility of each analyst.

Addressing the Challenges

Although the challenges described above are substantial, the CBC review panel concluded that steps can be taken to improve the scientific and conservation value of CBCs. It also strongly endorsed the view that the potential value of the data set justifies the effort and expense of taking those steps. Here, we provide a brief overview of the panel's primary recommendations to NAS. For details, further justifications, and additional recommendations, see Francis et al. (2004).

There are two avenues to enhancing the value of CBCs: minimizing limitations of the historical data set to allow extraction of maximum value from data collected in the past, and changing the survey protocol to improve the value of data collected in the future.

Making the Best Use of Historical Data

Collecting Better Data in the Future

Although it is theoretically possible to revise the CBC protocol to address all the challenges related to site selection and effort variation, the panel concluded, as others have previously (Hickey 1955, Bock and Root 1981), that changing survey design and data-collection methods to mold the CBC into a rigorous population-monitoring program would significantly reduce its value, including much of its scientific potential—primarily because participation would undoubtedly fall off drastically. Moreover, CBC sites have historical and local interest that make continued sampling at the existing sites desirable, and major change of survey design would interfere with comparability of future and historical data. Finally, important nonscientific benefits of the CBC would be lost, including social and educational benefits to participants and the community at large and value as an entry point for citizen science (Francis et al. 2004).

Nonetheless, the panel felt that some relatively small changes in CBC protocol, which would not much alter the CBC experience for participants, could contribute substantially to increasing the usability and scientific value of future data. We recognize that implementation of these recommendations, summarized below, will not resolve the fundamental sampling issues associated with site location and sampling within sites. If a high-quality winter monitoring program is needed, a wholly separate survey should be developed instead, with an appropriate statistical design.

Conclusions

In summary, the panel felt that the CBC constitutes a valuable data set with tremendous potential for additional scientific and conservation application. Nonetheless, the CBC has limitations associated with its design and orientation toward public participation that must be carefully considered and accommodated in any use of the data. Although analysis of existing data can be greatly improved using the best available methods, the very real limitations on inference imposed by the survey design must always be kept in mind. The NAS should not only provide access to the full data set and the best-analyzed results available, but should also provide users with a clear explanation of appropriate and inappropriate uses of those results. Christmas Bird Count field methods should not be altered to the extent of compromising comparisons of future with past data, but there are incremental improvements that can be made to address some of the CBC's scientific limitations. Other issues are not presently resolvable, but should be the subject of ongoing research and discussion to permit development of new approaches both to field protocols and data analysis.

The CBC review panel believes that implementation of its recommendations (Francis et al. 2004) will greatly enhance reliability of inferences based on CBC data, and ensure their optimal use for science and conservation activities. The recommendations are achievable, and the enhanced value and credibility of CBC should justify the effort and expense of carrying them out.